UNCLASSIFIED 


R.ep'uxtuced 
luf.  the 


ARMED  SERVICES  TECHNICAL  INFORMATION  AGENCY 
ARLINGTON  HALL  STATION 
ARLINGTON  12,  VIRGINIA 


UNCLASSIFIED 


NOTICE:  When  government  or  other  drawings,  speci¬ 
fications  or  other  data  are  used  for  any  purpose 
other  than  in  connection  with  a  definitely  related 
government  procurement  operation,  the  U.  S. 
Government  thereby  incurs  no  responsibility,  nor  any 
obligation  whatsoever;  and  the  fact  that  the  Govern¬ 
ment  may  have  formulated,  furnished,  or  in  any  way 
supplied  the  said  drawings,  specifications,  or  other 
data  is  not  to  be  regarded  by  implication  or  other¬ 
wise  as  in  any  manner  licensing  the  holder  or  any 
other  person  or  corporation,  or  conveying  any  rights 
or  permission  to  manufacture,  use  or  sell  any 
patented  invention  that  may  in  any  way  be  related 
thereto. 


BOEING 


SCI  ENTI  FIC 
RESEARCH 
LABOR  ATORI 


ES 


Studies  in  Generalized  Random  Walks 

II.  The  Expected  First  Passage  Time 


A.  William  Kratzke 


Mathematics  Research 


November  1962 


Dl-82-0213 


STUDIES  IN  GENERALIZED  RANDOM  WALKS 
II.  The  Expected  First  Passage  Time 


by 


A.  William  Kratzke 


Mathematical  Note  No.  279 
Mathematics  Research  Laboratory 
BOEING  SCIENTIFIC  RESEARCH  LABORATORIES 


November  19&2 


Abstract 


A  particle  moves  along  a  line  in  steps  subject  to  the  following 
probability  laws  the  particle  moves  from  x  to  +  C^x  with  proba¬ 
bility,  p  or  from  x  to  C^x  with  probability,  q(=  1  -  p) .  Such  a 
movement  characterizes  a  type  of  random  walk  and,  as  such,  poses  prob¬ 
lems  associated  with  random  phenomena.  One  such  problem  is  that  of 
finding  the  expected  number  of  steps  necessary  for  a  particle  to  cross 
a  pre-assigned  level  for  the  first  time,  having  taken  its  first  step 
from  the  initial  position,  x.  This  expectation  is,  of  course,  a 
function  of  x  and  is  designated,  D(x). 

The  properties  of  the  function,  D(x) ,  are  studied  and  put  to  use 
in  determining  D(x)  explicitly.  Since  the  function  depends  strongly 
on  the  choice  of  the  constants,  C 2  and  C^,  the  problem  must  be 
treated  by  cases  which  are  characterized  by  the  relationship  between 
and  C^.  The  ease  with  which  solutions  are  found  also  depends  on 
the  position  of  the  absorbing  barrier  and  several  choices  are  studied. 


This  paper  is  a  sequel  to  the  paper,  Studies  in  Generalized  Random 
Walks,  I.  Distribution  Functions  and  Moments,  Mathematical  Note  No.  27 7. 
A  brief  review  of  the  kind  of  random  walk  treated  in  these  studies 
seems  in  order  here. 

Consider  the  motion  of  a  particle  in  an  interval  governed  by  a 
fixed  probability  law.  The  particle  position  after  n  steps  is  given 
by 

+  ^2Xn  1  w^'*1  probability  p 

x  = 
n 

^3Xn  1  with  probability  q(=l  -  p) . 

It  is  easy  to  see  that  the  particle  is  constrained  to  motion  in 
C1 

the  interval,  [0,q - -  ] .  Having  begun  its  walk  in  that  interval, 

1  "  C2 

it  can  never  leave. 

An  absorbing  barrier  at  some  point  in  the  interval,  say  y,  ter¬ 
minates  the  'walk.'  This  means  that  if  x^,  the  initial  point,  is 
greater  than  y,  the  walk  never  starts  and  if  x^  is  less  than  y, 
the  motion  stops  when  the  particle  first  passes  y.  Then,  given  an 
initial  position,  x^,  and  an  absorbing  barrier,  y,  it  is  natural  to 
inquire  into  the  problem  of  finding  the  number  of  steps  necessary  for 
the  particle  to  pass  y.  This  number  is  a  random  variable  and  finding 
its  expected  value  is  the  object  of  this  paper.  If  each  step  takes 
unit  time,  the  problem  is  a  first  passage  time  or  absorption  time 
problem.  Similarly,  if  the  process  is  regarded  as  a  game  (such  as  the 
gambler's  analogue  discussed  in  I),  the  absorption  time  is  just  the 
duration  of  the  game . 


II.  Expected  First  Passage  Time 


1.  General  expressions  and  properties. 


The  most  general  expression  for  the  expected  first  passage  time 
may  be  derived  by  examining  the  simple  one-step  process.  If  x  is 
the  starting  point,  the  next  point  is  either  +  C^x  or  C^x  with 
probabilities  p  and  q  respectively.  Let  y  be  the  coordinate  at 
which  there  is  an  absorbing  barrier  and  let  D(x)  be  the  expected 
time  or  number  of  steps  to  cross  y  from  below,  having  started  at  x. 
We  assume  that 


0  <  y  < 


1  -  C  ’ 

2 


D(x)  <  CD  , 

D(x)  =  0  when  x  >  y  and 
DCx^  >  D(x2)  if  x1  <  x2- 

Then,  starting  at  x,  if  the  first  step  is  to  +  C2x,  the  process 
continues  as  if  +  C2x  were  the  initial  position  and  if  the  first 
step  is  to  C^x,  the  process  continues  as  if  that  were  the  initial 
position.  Therefore,  D(x)  may  be  written  in  terms  of  +  C2x) 

and  D(C^x) . 


D(x)  =  pWC^  +  C2x)  +  1)  +  q(D(C^x)  +  1) 
=  pD(C^  +  C2x)  +  qD(C^x)  1. 


(1) 


This  is  the  basic  equation  for  these  absorption  time  analyses. 
However,  there  are  several  other  expressions  which  prove  useful. 
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p 

If  (1)  is  applied  successively  to  D(C^x) ,  D(C^x) ,  etc.,  D(x) 
may  be  written  as 

D(x)  =  pD(C^  +  C^x)  +  qD(C^x)  +  1 

=  pD(C^  +  C^x)  +  q[pD(C^  +  C2C^x)  +  qD(C^x)  +  l]  +  1 

=  pD(C1  +  C2x)  +  pqD(C1  +  C2C5x)  +  q2[pD(C1  +  C^x)  +  qD(C^x)  +  l]  +  1  +  q 

=  pD(C^  +  C2x)  +  pqD(C^  +  C2C^x)  +  pq2D(C^  +  C2C2x)  +  ... 

+  pq^D(C^  +  C.^C^x)  +  ...  +  1  +  q  +  q  +...+q  +  •  •  • 

CD  CD  , 

=  p  E  qu(C  +  C_C  x)  +  E  q 
k=0  1  ^  3  k=0 

(2) 

=  p  E  qKD(C..  +  C5C^x)  +  ±  . 
k=0  x  d  3  P 

Note  that  +  C2x  may  be  greater  than  y  or,  more  generally,  it 

may  be  that  C..  +  C_x,  C.  +  C_C,x,  ...  C,  +  C_.c"!x  >. y.  rfhen  the  first 
i-  d  . id}  Xd} 

m  +  1  terms  of  the  sum  in  ( d )  contribute  nothing. 

Another  expression  for  D(x) ,  similar  to  (2) ,  may  be  found  by 

2 

successive  applications  of  (1)  to  D(C^  +  C^x) ,  D(C^  +  C^C2  +  C^x) ,  etc. 

D(x)  =  pD^  +  C2x)  +  qD(C  jX>  +  1 

=  pfpDCC-j^  +  +  C2x)  +  qD(C^C1  +  C^x)  +  l]  +  ^D(C^x)  +  1 

=  p2[pD(C1  +  0^2  +  C^2  +  C2x)  +  qD(C^C1  +  +  C^C2)  +  l] 

+  qpD(C^C^  +  C^x)  +  qD(C^x)  +  1  +  p 
=  qD(C^x)  +  qpD(C  (Cx  +  C^x))  +  qp2D(C7.(C;L  +  +  C2x))  +  ... 

1  -  ck 

+  qp  DCC^^-y— +  C^x))  +  l+  p  +  p^+...+  p^.  (}) 


4 


Having  thus  established  the  functional  relations,  (l) ,  (2),  and 
(4) ,  some  general  properties  of  the  expected  duration  function  can  now 
be  derived. 

It  is  expedient  to  introduce  here  the  operator  notation  used  in  I 
The  operators,  T^,  T^,  and  T  are  defined  by 

Tlx  =  C1  +  C2X 

T_x  =  C  x 
^  3 

Tx  =  either  T^x  or  T^x. 
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One  obvious  property  of  these  operators  is  nonlinearity.  To  be  sure, 
is  a  linear  operator  but  any  combination  of  operations  that 
includes  at  least  one  operation  is  nonlinear. 

Tm(x1  +  x2)  <  T1"^  +  Tmx2,  (5) 

where  at  least  one  of  the  operators  is  a  T^.  Now  suppose  t"1  x  =  y. 

Then  Tm(x  +  Ax)  >  y  and  one  is  led  to  suspect  that  the  number  of  steps 
necessary  for  a  particle  to  go  beyond  y,  having  started  at  x,  is 
surely  at  least  one  greater  than  the  number  necessary  if  the  particle 
had  started  at  x  +  Ax.  In  particular,  if  only  forward  steps  are  con¬ 
sidered  (assume  that  the  probability  of  a  T^  operation  is  one) ,  then 
(5)  implies  that  the  expected  duration  would  have  discontinuities  at 


*-Cl 
°2  ’ 


y  -  Cjd  +  C2) 

4 


,  etc.  and  the  jumps  would  be  of  unit  ampli¬ 


tude.  That  there  are,  in  general,  discontinuities  in  D(x)  is 
demonstrated  in  the  following  theorem. 


Theorem  1 . 

C!  >  0,  0  <  C2,  C5  <  1. 

There  exists  discontinuities  in  the  expected  duration  function. 

Proof:  First  note  that  the  point,  y  (the  absorbing  barrier  point)  is 
a  point  of  discontinuity. 

D(y)  =  1  +  qD(C^y) 

D(y+)  =  0 

D(y)  -  D(y(+))  =  1  +  qDtC^y)  >  0, 


(6) 
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since  D(x)  >  0  for  all  xe[0,  - - 1  •  Similarly, 

i  -  c2 

y  -  c  y  -  c. 

D(— -— )  =  PD(y)  +  qD(C5  —■  -)  +  1 


2 

y  -  Gn 


'i  y  -  ci 

D( — - — -  +)  =  1  +  qD(C7  —  -  +  ) 


'3  c. 


(7) 


y  -  c  y  -  c  y  -  e  y  _  c 

D( — — ~)  -  D ( — — — —  +)  =  qr D(C  -  r  ■  )  -  D(C?  ~  ~  +)]  +  pD(y)  . 

C2  C2  5  °2  5  °2 


Continuing  in  this  way,  it  is  easy  to  show  that  many  more  such  discontinuities 

exist.  In  fact,  there  is  such  a  discontinuity  at  each  of  the  points, 

1  - 


(1  -  <£) 


ln(l  - 


-y) 


1  2.'  r  11 

^  -  cid  -  cj)* k  =  °’  2-*-n’ where  n  =  L - isc: — J 


1  -  c. 


ln(l  - 


(read  greatest  integer  in 


InC. 


). 


That  there  may  be  other  points  of  discontinuity  is  also  easy  to  see. 

y  -  C 

For  example  if  C7y  >  — -z - ,  then  surely  there  is  a  discontinuity  at 

y  -  C  5  2  y  -  C 

r-s —  generated  by  the  one  at  — p - ,  which  in  turn  is  generated 

°2C3  2 

by  the  discontinuity  at  y.  This  completes  the  proof  of  the  Theorem. 


Extending  the  results  of  Theorem  1  provides  some  coarse  bounds 
on  the  functions  at  the  points  of  discontinuity.  We  will 
cite  only  two  examples.  Consider  equation  (5).  Because  of  the 
monotonicity  property, 

D(y)  =  1  +  qD(C^y)  >  1  +  qD(y) 

D«  >  5  • 


(8) 
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Again  using  the  mono.tonicity  of  D(x)  and  (6), 


y  -  c  y  -  c 

D(~— --)  =  pD(y)  +  qD(C,  —  — )  +  1 
C2  '  C2 
y  -  c, 

>  pD(y)  +  qD( — p — -)  +  1 

C2 

y  -  ci  ip 

>  D(jr)  . 


(9) 


Furthermore,  it  can  be  shown  that 

(1  -  c*) 

y  '  °1  (1  -  C  ) 

D< - g - . 


(10) 


The  inequality  (10)  gives  bounds  for  the  values  of  the  function 
at  discontinuity  points.  In  addition,  something  of  the  nature  of  the 
jump  magnitudes  can  be  determined  using  the  same  ideas.  Consider  the 

y  -  ci 

saltus  at  — r -  . 


DC- 


y  -  c, 


-)  -  D(- 


2 

y  -  c, 


+)  =  p(D(y)  -  D(y+) )  +  q(D( 


C  (y  -  C  )  C  (y  -  C  ) 

- k)  _  d(—2 - 


+) 


2 

y  -  c. 


Now,  if  C  (- 
y  -  c,  3 


-)  is  not  a  point  of  discontinuity,  the  saltus 


'i  y  -  c1 

at  — p -  is  just  p  times  the  saltus  at  y.  Whether  or  not  C  ( — p - 

2  3  2 
is  a  discontinuity  point  is  determined  by  the  constraints  involving 

C_  and  C,..  The  cases  C_  +  C,  <  1  and  C_  +  CT  >  1  must,  in  gen- 

2  5  2  5  -  2  5 

eral,  be  handled  separately.  In  an  earlier  paper  (Studies  in  Generalized 

Random  Walks,  I.  Distribution  Functions  and  Moments)  it  was  shown  that 

if  C_  +  C_  <  1,  the  path  that  carries  a  particle  from  x  to  y  is 

2  5“  „  r 


) 


y  -  c., 


unique.  Therefore  C^(-  ^ 


-)  is  not  a  point  of  discontinuity 


(11) 
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(by  point  of  discontinuity  is  meant  one  of  the  points  reached  by  starting 

y  -  cx  y  -  c1 

at  y  and  using  inverse  transformations,  — -z - ,  r-r; - ,  etc.)  and  the 

2  3°2 

second  term  on  the  right  of  (11)  vanishes.  Similarly, 

y  -  C.  y  -  C 

D(-5T — )  -  D(  ■  -  +)  =  qp(D(y)  -  D(y+))  =  qpD(y) .  (12) 

3  2  3  2 

Having  established  the  fact  that  D(x)  has  discontinuities,  it 
remains  to  determine  the  nature  of  the  solutions  between  discontinu¬ 
ities.  One  is  strongly  led  to  suspect  that  the  function  remains 
constant  there.  Heuristically,  the  argument  for  this  is  that  any  path 
that  carries  x  beyond  y  but  does  not  carry  a  point  in  a  neighbor¬ 
hood  of  x  beyond  y  must  necessarily  have  an  infinite  number  of 
steps  or  one  of  the  two  points  would  be  a  point  of  discontinuity.  The 
paths  with  an  infinite  number  of  steps  contribute  nothing  to  the 
expected  value  since  they  have  zero  probability.  Theorem  2,  below, 
demonstrates  this  more  rigorously. 

Theorem  2.  If  the  expectation  function  has  a  derivative  at  x,.the 
derivative  vanishes  there. 

Proof:  If  D'(x)  exists,  (l)  may  be  differentiated  and  D' (x)  must 
then  satisfy 

D’(x)  =  pC_D' (C_  +  C.x)  +  qC,D' (C,x) .  (13) 

^1233 

The  derivates  of  D(x)  are 

_  D(x1)  -  D(x)  _  D(x  )  -  D(x) 

Urn  - ,  Urn  - , 

X-^X  X1  "  X  x^x  X1  ~  X 

JM_  D(V  -  DM.  and  Jin.  ^ 

X1^X  X1  ”  X  Xl|X  X1  - 


X 
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Note  that  each  of  the  derivates  satisfies  (13)*  even  at  the  discon¬ 
tinuities,  but  at  those  points  they  are  not  necessarily  equal  so  the 
derivatives  do  not  exist  there.  Let  D' (x)  be  one  of  the  derivates 
and  define  M  and  6  as 


m 

M  =  {x|D'(x)  >  -m}  and 
m 

6  =  inf  D*(x). 


(15) 


M 

Choose  x  such  that 

€ 

D'(xe)  <  6  +  e. 


Then 

pC  D' (C..  +  C_x  )  +  qC,D' (C,x  )  =  D' (x  .  )  <  6  +  *  <  D' (C.  +  C_x  )  +  e, 
2  12e^3  3e  e  l2e 

qC^D'  (C^x^ )  <  (1  -  pC2)D’(C1  +  C^)  +  e  (16) 


pC_D' (C  +  Cnx  )  +  qC-D' (C,x  )  =  D' (x  )  <  6  +  e  <  D'(C  *  )  +  e 
2  1  2  e  33®  *  5  * 


pCgD'Wj  +  C2xe)  <  (1  -  qC3)D'(C5X#)  +  «  (17) 

PCp  . 

t - ~r~V  (C,  +  C_x  )  <  D'  (C  x  )  +  r-~r-  • 

1  -  qC^  1  2  e  3  *  1  - 

VCp 

Multiply  both  sides  of  (l6)  by  ^  ^  ■  and  then  apply  (17). 

pC  qC,  pC  pC  e 

(1  -  pC2)(l  qC5)  D'(C3Xc)  <  (1  -  qC^)  D'(C1  +  C2Xe)  +  (1  -  qC3)(l  -  pC2) 

<  D'(C5xe)  +  (1  _  qC5)(l  -  pC2)  ’ 


(PC_  +  qC  -  1)D' (Czx )  <  e . 

2  3  3  c 


(18) 
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qC 


The  same  procedure  using  - - 57-  as  a  multiplier  on  (17)  yields 

1  - 

pC-qC,  qC,  *  <lC,e 

(1  -  pC2)(l  -  qC5)  D'  (ci  +  C2x«)  <  (  1  -  pC2)  D'  (C3xe)  +  7T  -  qC^Hi  -  pc2). 


<  D'(C1  +  C2xg)  +  (1  r  qC j ) ( 1  -  pC2) 

(pC2  +  qC  -  1)D'(C1  +  C2xe)  <  «. 

Since  p  +  q  =  1  and  0  <  <  1,  pC2  +  qC^  -  1  <  0.  The 

monotonicity  of  D(x)  insures  that  D* (x)  <  0.  Therefore 

(pC_  +  qC_  -  1)D' (C  x  )  >  0 
2  3  3  e  - 

(pC.  +  qC  -  1)D* (C_  +  C.x  )  >  0. 

<l  3  i  c.  e  — 

Since  r  was  chosen  arbitrarily  and  D'(x)  may  be  any  of  the 
derivates,  the  conclusion  is  that 
6=0. 


(19) 


(20) 


(21) 


Finally,  the  set  of  x  for  which  (21)  is  true  is  simply 

{x|D'(x)  =  0)  =  UM  (22) 

m  m 

The  usefulness  of  Theorem  2  depends,  of  course,  on  the  existence  of 
derivatives.  The  next  theorems  demonstrate  the  existence  of  right- 
hand  and  left-hand  derivatives. 


Theorem  }a.  The  right-hand  derivative  of  D(x)  exists  for  all  x. 


Proof:  Consider  the  first  and  third  derivates  of  (14) .  jSach  of  these 

satisfy  (13)  and  is  0  or  -<d  .  If  A  is  the  difference  quotient,  it 

is  only  necessary  to  show  that  limA  <  limA ,  since  the  other  inequality 

is  obvious.  Take  a  point,  x,  in  the  interval  and  choose  a  sequence,  g, 
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which  goes  to  zero  from  above  (^  \  0) ,  such  that 

n  1 

D(x  +  ?  )  -  D(x)  _  D(x..)  -  D(x) 

lim  - yS - -  _  Hj,,  - ± -  =  o.  (23) 

n-KD  x^x  1 

Choose  a  second  sequence,  ^  Oi  such  that 

D(x  +  ti  )  -  D(x)  D(x. )  -  D(x) 

lim  - —  =  Urn  -  =  0.  (24) 

n-KD  'n  xlfx  ^ 


(25) 


(26) 


which,  since  limA  >  limA ,  demonstrates  that 


_  D(x.)  -  D(x)  D(x  )  -  D(x) 

lim  -  =  lim  -  (27) 

X,  -  X  -  X-  -  X 

Xi|X  1  xltx  1 

and  the  theorem  is  proved. 


Theorem  3b.  The  left-hand  derivative  of  D(x)  exists  for  all  x. 

Proofs  The  proof  follows  exactly  the  proof  for  Theorem  3s • 

The  last  general  property  considered  is  that  of  the  uniqueness  of 
the  solution.  Assuming  the  existence  of  functions  which  satisfy  (1), 


Choose  a  subsequence  ?  such  that  y  >  r^. 

°k  \ 

Th"”  *,  ♦  o  -  dm  ♦  y  -  D(*> 

o  >  — — ■ — - > - - - 


S  D(x  +  *!  )  -  D(x)  D(x  +  S„  )  -  P(x) 


Combining  (23),  (24),  and  (25) 


_  D(x  )  -  D(x)  D(x  )  -  D(x) 

lim  -  <  lim  - , 

.  X,  -  X - —  X,  -  X 

Xpfx  1  Xif X  1 


1 
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(2) ,  and  (4) ,  the  following  theorem  demonstrates  that  there  iB  only 
one  such  solution. 


Theorem  4.  Let  D^(x)  and  D2(x)  be  two  solutions  of  (l) . 

If  6(x)  =  D  (x)  -  D  (x)  and  6(x)  =  0  for  x  >  y,  then 

C1 

£>(x)  =  0  for  all  xe[(),  r - pp-].' 

1  ‘  °2 

Proof:  Let  x^  be  a  point  at  which  6(x)  assumes  a  maximum  value.  Then 


pb(C1  +  C^)  +  qb(C^x^)  =  bCx^)  >  6(C1  +  C^) 
qb(C^x^)  >  (1  -  p)6(C1  +  C2x1) 


(28) 


6(0^)  >  6(Cl  +  C^). 


Similarly, 


p6(C1  +  C2x1)  +  qb(C^x^)  =  oCx^)  >  6(0^) 


+  C2x1>  >  6(0^) 


and,  therefore 


6(C1  +  C^x^)  =  6(0^)  =  6(x1). 


(29) 


(30) 


Then,  if  x^  is  a  maximum  point,  +  C2x^  and  C^x^  are  maximum 

2  3 

points.  Furthermore  T^x^,  T^,  etc.  are  also  maximum  points,  and  for 


all  k  greater  than 


c  c 

ln(l  -  C  '  y^  ■  ln^l  -  C  ■  X1^ 

r - i - £ - 1 

L  InC,  J ’ 


T1X1  >  y’ 

By  hypothesis, 

6(tJxx)  =  0, 

so  the  conclusion  is  that 


max 


b(x)  =  0. 


0  <  x  < 


-  -  1  -  C_ 


(3D 
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A  similar  argument  may  be  used  to  show  that 

<1 


min  „  6(x)  =  0, 


0  <  x  < 


- 

so  that,  finally 

C, 

6(x)  =  0  for  all  xe[0,  •: - s~3 . 

1  '  C2 


(32) 


(33) 


Although,  the  properties  of  D(x)  set  forth  in  the  preceding 
theorems  describe  the  general  behavior  of  the  function,  there  is  no 
method  of  solution  yielding  a  function  that  satisfies  (1),  (2),  and 
(4)  for  arbitrary  values  of  the  constants,  C^,  C^,  and  C^.  Therefore, 
it  is  necessary  to  treat  cases  separately,  each  case  being  characterized 
by  a  different  constraint  on  the  constants. 

The  first  two  cases  perhaps  should  not  be  part  of  this  discussion, 
since  the  constant,  ,  is  taken  to  be  zero  and  one  for  the  two  cases, 
and  this  violates  the  general  constraint,  0  <  C^,  C 2,  <  1.  However, 

the  solutions  are  limiting  cases  and,  as  such,  give  some  insight  into 
the  kind  of  solutions  to  expect. 


let 


When  =  0,  equation  (l)  becomes 

D(x)  =  pD(C1  +  C^x)  +  qD(0)  +  1. 

C1  C1 
ln(1  _  g - y)  -  ln(-j-  "c - x) 


W 


k  =  [- 


InC. 


-1  ♦  1. 


Applying  (J4)  to  itself  k  times  yields,  for  D(x) 

1  -  Ck 

D(x)  =  pkD(C^  y  _  q  +  Ckx)  +  (1  +  qD(0))(l  +  p  +  p^  +. . .+  pk  ^) , 


(35) 


Since  D(x)  =  0  for  x  >  y,  the  first  term  on  the  right  vanishes. 

,  k 

D(x)  =  —  p-(qD(0)  +  1).  (36) 

q 

Let  r  be  the  least  number  of  forward  steps  (x  -*  +  C^x)  necessary 

to  carry  a  particle  from  zero  to  a  point  beyond  y. 


Then 


r  =  r 

D(0) 

D(0) 


y)  -  ln(j 

lnC2 


is  given  by 


r 


qp 


1  +  1. 


(37) 


and  finally,  D(x)  becomes 
t  k 

D(x)  =  ■  ~  f  (qD(0)  +  1) 

=  +  1)  (38) 

q  r 

p 

,  k 

=  . 
r 

qp 

The  other  limiting  case  of  interest  is  that  one  for  which  =  1. . 
Here  a  particle  in  a  one-dimensional  random  walk  never  moves  backward, 
but  may,  with  probability  q,  remain  at  its  present  position.  The 
problem  of  finding  the  expected  number  of  steps  until  first  passage 
may  be  solved  directly  because  the  first  passage  distribution  is  deriv¬ 
able  and  the  sum  defining  the  first  moment  is  expressible  in  closed 
form.  Later  on,  the  same  result  will  be  obtained  using  the  functional 
equation  (l) . 

Let  k  be  the  least  number  of  forward  steps  necessary  to  carry  a 
particle  from  the  point,  x,  to  a  point  greater  than  y.  This  is  the 
same  k  as  the  one  used  in  the  previous  problem.  If  the  number  of 
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steps  necessary  to  cross  y  for  the  first  time  having  started  at  x, 
is  N,  and  P(i)  =  Pr{N  =  i} ,  then 

P(k)  =  pk.  (59) 

For  N  =  k  +  1,  consider  the  paths  that  have  k  +  1  steps,  start  at  x, 
and  terminate  beyond  y.  Initially,  and  after  each  forward  step,  one 
move  that  goes  from  x  to  x(x-»C^x  =  l)  may  take  place.  There  are 
just  k  ways  for  this  to  happen  so 

P(k  +  1)  =  kqpk.  (bO) 

Recall  that  the  number  of  ways  one  can  put  C  indistinguishable  ob¬ 
jects  into  M  numbered  boxes  With  no  restrictions  on  the  number  of 

objects  in  any  one  box  is 

(M  +  C  -  1)1 
(M  -  1) 1C!  * 


If  the  objects  correspond  to  the  steps,  x  -•  C^x 
correspond  to  forward  step  positions  given  by 


+  C2X' 


i 


1,  2, * • *k, 


x  and  the  boxes 


then  the  number  of  ways  that  the  particle  can  experience  n  'standing 
still'  steps  at  k  positions,  with  no  restriction  on  the  number  at 

each  position  is 

(k  +  n  -  1)1 
(k  -  1) 1 n!  * 

Therefore , 


P(n  +  k) 


( k  +  n  -  1)1  n  k 
(k  -  l)!ii!  q  P  * 


(41) 


l6 


Then,  it  remains  but  to  multiply  (4l)  by  (n  +  k)  and  sum  over  all  n 
to  find  D(x).  If  N  is  the  first  moment  of  N, 

CD 


D<*>  - 5  -  I  VAnHI'*"  *  k>’r 


X 

n=0 

CD 

k,  V  (n  +  k)  I  n 
=  p  k  L  “kin! 
n=0 


(42) 


JS£_ 


(l  -  q) 


k+1  ”  p* 


The  other  method  for  obtaining  the  solution  (42)  involves  use  of 


(43) 


the  basic  equation  (1).  When  =  1,  that  equation  becomes 
D(x)  =  PD(C1  +  C£x)  +  qD(x)  +  1 
D(x)  =  D(Cr  +  C  x)  + 

1  2  p 

Again,  one  applies  (43)  to  itself  k  times,  i.e.,  until  the  argument 

of  the  first  term  on  the  right  exceeds  y. 

,k 


D(x)  =  D(C 

k 
“  • 

P 


i  -  c: 

ll-c/  C2x)  +  p 


(44) 


—  •  • 


As  expected,  the  results  of  (42)  and  (44)  agree. 


These  limiting  cases  are  useful  when  only  bounds  are  needed  for 

the  function.  Note  that  for  any  combination  of  constants  subject  to 

the  condition,  0  <  C^,  C^,  <  1,  D(x)  is  bounded  above  and  below 

by  the  solutions  for  C,  =  0  and  C,  =  1. 

3  3 

-  <  D(x)  <  -  ~-gk. 

P  r 

qp 


(45) 


17 


Considered  next  are  the  solutions  of  (1)  when  <  1.  In 

the  interests  of  simplicity,  let  the  absorbing  barrier  first  be  taken 
as  one  of  the  points  reached  by  forward  steps  only,  having  started  at 
the  origin,  i.e. 

(i  -  C*) 

y  =  C,  — - - —  for  some  k. 

1  1  -  c2 

First,  let  y  =  C^.  Then  for  all  x  in  the  interval  (0,C^] 

D(x)  =  1  +  qD(C^x) 

=  1  +  q(l  +  qD(C jX) ) 

=  1  +  q  +  q^  +  . . .  +  qnD(C^x) . 

Since,  by  hypothesis,  D(0)  <  cd  ,  the  last  term  vanishes,  and 


03 


D(x)  =  S  q1  =  i  . 

.  „  p 

i=0  * 


(46) 


From  (1) , 

D(0)  =  D(C1)  +  i,  (47) 

and  (1)  is  then  solved  for  all  xefO,^]. 


The  solution  (46)  is  available  immediately,  if  it  is  recognized 

that 

D(x^)  =  D(x2) ,  0  <  x^ , x^  <  C^.  (48) 

To  see  this,  note  that  every  path  that  carries  x^  beyond  y  also 
carries  x2  there.  Note,  also,  that,  in  this  one  instance,  it  matters 
not  what  the  constraint  relating  C2  to  is,  just  so  long  as 

0  <  C  <  1.  If  (48)  obtains, 

D(x)  =  1  +  qD(CjX)  =  1  +  qD(x) 

D(x)  =  i, 

and  then  (46)  and  (48)  agree. 


(49) 
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Next,  let 
D(x) 


y  =  C^(l  +  C^) .  When  x  is  in  the  interval  (0,C^] 

=  pD(C1  +  C2x)  +  qD(C^x)  +  1 

=  p^C^l  +  C2)  +  C2x)  +  pqD(C5(C1  +  C2x)  +  qDCC^x)  +  1  +  p. 


=  0  +  pqD(C  (C  +  C  x))  +  qD(C  x)  +  1  +  p.  (50) 

Using  the  same  method  as  that  used  in  the  previous  problem,  i.e. 
iterating  (50) 

D(x)  =  pqD(C^(C^  +  C2x))  +  qD(C^x)  +  1  +  p 

=  p2q2D(C5C1(l  +  CgC  )  +  C2D2x)  +  pq2D(G2(C;L  +  C2x)) 

+  pq2D(C,(C1  +  C,C *))  +  q2D(C2x)  +  (1  +  p) (l  +  q  +  pq) 


=  (1  +  p)(l  +  (1  -  p2)  +  (1  -  p2)2  +  ...) 


(51) 


CD 


-  (l  +  P)  £  (l  -  p2)1 


i=0 


=  (1  +  p) 


1  -  (1  -  p  ) 


_  i_±_E 

-  2 
P 

=  i  +  ±. 

P2  P 

Again,  the  finiteness  of  D(x)  assures  the  vanishing  of  all  but  the 
last  term  in  iterative  scheme.  If  x  is  in  the  other  interval, 

(c1,c1(i  +  c2)l, 

D(x)  =  1  +  qD(C^x) . 


:i-9 


But  C^x  <  C^.  So 

D(x)  =  1  +  q(-^  +  i) 

d  p 
P 

=  1  +  JL(  1  +  p)  (52) 

P 

_  !  +  1  -  P2  _  i 

P  P 

Note  that  the  argument  following  (48)  applies  here,  too.  However,  the 
constraint  <  1  is  necessary  so  that  C^x  <  for  all  x 

in  [0,y].  Following  the  method  for  obtaining  (49), 

D(x)  =  pqD(C^(G1  +  C2x))  +  qD(C^x)  +l+p  0  <  x  <  (^ 

1+P  1  +  p  1  1 

1  -  pq  -  q  2  2  p 


Of  course,  the  same  thing  applies  for  xe(C^,C^(1  +  0o)].  Now,  one 
can  generalize  for 


First  let  xe(0,C^].  The  method  is  as  follows:  iterate  the  first  term 
of  (1)  until  the  argument  of  the  first  term  is  greater  than  y. 

D(x)  =  pDtC^  +  C2x)  +  qD(C^x)  +  1 

=  p2D(C1(l  +  C2)  +  C2x)  +  pqD(C  (Cx  +  C2x))  +  qD(C^x)  +  1  +  p 
1  -  f 

=  pkD(C1  — - +  C^x)  +  qlpk  1d(\_i^  +  pk  2D(x1c _2)  +  ■••  +  °(C^x 

+  1  +  p  +  p2+...+  pk_1,  (54) 

where  0  <  x±  <  C  for  i  =  1,2, ...k  -  1.  If  we  designate  C^x  by  xQ, 
a  more  compa.ct  form  for  (55)  is  available.  In  fact,  a  special  case  of 
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equation  (4)  obtains.  The  first  term  on  the  right  side  of  (53) 
vanishes  and 


k-1  .  k-1  . 

D(x)  =  q  S  p1D(x. )  +  Sp 
i=0  1  i=0 

=  q^p^Cx.)  ♦  . 


Again,  remembering  that  D(x).  <  cd  ,  iterating  (54)  yields 
D(x)  =  (  1  +  (1  -  Pk)  +  (1  -  pk)2  +  ...) 

q 


(l  -  p)pk 

-i  2  1 

1  +  p  +  p  +  . . .  +  P 
k 
P 


(55) 


(56) 


A  simpler,  more  direct  solution  is  again  available  if  the  same  heuristic 
argument  is  used  as  that  used  for  finding  (49)  and  (52) .  All  'the 
arguments  in  the  sum  in  (54)  are  in  the  interval  (0,^]  so  we  may 
equate  all  of  the  functions  D(x^)  to  D(x) . 


k-1  k 

D(x)  =  D(x)q  E  pi  +  1  —  ^ 

i=0  q 

i  k  i 

D(x)  =  i-=_R 

q 


i  k 

i  -  q(^=-E-) 
q 


(57) 


*| 

_  l  --E- 

k  ’ 

qp 

which  agrees  with  (55)*  For  xe 


f„  Cf1  1  -  C2  1 

VCi  S  1  -  C,  '  n  2,3,  • 


•  •  k  9 
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the  first  and  second  forward  step,  starting  at  the  origin,  i.e., 

<  y  <  0^(1  +  C^).  become  evident  later  on  that  a  further 
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subdivision  on  the  values  y  may  assume  is  necessary  and  to  that  end 
the  following  inequality  must  be  proved. 


°i<rrsx<ci(1*<y- 


(60) 


Since  both  C 2  and  are  less  than  one,  the  left  side  of  the 

inequality  is  obvious.  We  need  only  the  constraint,  <  1  to 

show  the  other. 

* 

C1  .  °1 
1  -  c2c3  ^  1  -  c2(i  -  c2) 

=  cx(i  +  c2(i  -  c2)  +  c2(i  -  c2)2  +  0^(1  -  c2)5  +  ...) 
=  c^i  +  c2)  +  cx(-  c2  +  s  c2(i  -  cp1) 

C2(l  -  C  )2 

=  C^l  +  c2)  ♦  c1(-  c|  +  r-"c(i-  c  )} 

c3  2  2 

“  Cia  +  °2)  '  1  -  0,(1  ♦  o 


<  c^U  +  c2). 


First,  take  the  case, 
C 


1  -  C2C5  <  y  ^  +  C2^ 


(61) 


From  (6l) ,  we  note  that 


y  -  C1  y  -  C 

-C—  =  -0—  *  V  -  V 


J  -  =1  -  C20,y 

- - - 1—  + 


Cf 


y(l  -  C2C  )  -  C 

_  - —2 -  + 


V 


c  -  c 

>  — c—  +  C5y  =  C5y. 


(62) 
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The  following  sketch  is  an  aid  to  remembering  the  ordering  of  pertinent 
points  t 
I — 


T 


°3y 


— r~ 

y  -  c, 


T 


T 


1  “  C2C3 


1 - 

y 

y  -  c. 


i 


Cl(l  +  C2) 


(63) 


Now,  to  return  to  the  problem.  When  0  <  x  <  — = - , 

"  °2 

D(x)  =  pD(C1  +  G2x)  +  qD(C^x)  +  1 

=  pqD(C^(C1  +  C2x))  +  qD(C^x)  +  1  +  p. 

From  (62)  and  the  condition  on  x 

y  -  C 

C3(C1  +  C2x)  -  C3y  -  "  c""~’ 

so  the  arguments  of  both  of  the  D(0  functions  on  the  right  side  of 
(63)  are  to  the  left  of  x.  The  solution  of  (63) then  reduces  to  that 
of  (53)  and 

(64) 

P  p" 

y  -  ci 

When  — r -  <  x  <  y, 

C2 

D(x)  =  1  +  qD(C^x) 


D(x)  =  i 

P  P2 


But 


so 


t^x  <  Cjy  < 


y  -  c. 


D(x)  =  1  +  q(i  +  \)  = 


(65) 
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The  remaining  part  of  the  interval,  (C^,C^(1  +  cannot  be 

taken  as  the  set  from  which  possible  values  for  y  may  be  drawn  and 

get  a  single  result  for  D(*)-  To  see  this  note  that  different  points 

Of  discontinuity  (relative  to  y)  arise  when  different  values  of  y 

C1 

are  taken  from  (C,  ,  - - „  -77-]  .  (The  latter  set  is  simply 

1  1  "  C2C3 

C, 

(C^CL  +  C2)]  -  (-  C^l  +  C2)].) 

2  3 

The  method  for  finding  the  values  of  y  which  admit  of  a  single  solu¬ 
tion  (in  terms  of  y)  is  to  determine  the  number  of  points  of 
discontinuity  to  allow  and  then  find  the  range  of  y  which  yields 
that  number  of  such  points.  Then  by  appealing  to  Theorems  2  and  3» 
we  may  take  D(*)  to  be  constant  in  each  interval  between  adjacent 
points  of  discontinuity.  For  example,  if  only  three  regions  are 
wanted,  proceed  as  follows:  Take  y  as  the  first  point  of  discon- 

y  -  C1 

tinuity,  and  since  y  >  CL  ,  surely  — ^ -  is  also  such  a  point 

1  C2 

(Theorem  1).  The  point, 

n  y  -  c  y  -  c 
T  1( - =) - - 

2  °2  C2C3 

may  be  the  last  point  of  discontinuity  or  it  may  generate  more  points. 
y  -  ci 

To  insure  that  _  _ —  is  the  last  such  point,  neither 

C2C3 

_i  y  -  ci  y  -  c, 

T-,  (-^-£ — )  .nor  T,1(-t;  ) 


ICC 
2  3 


2  C2C3 


may  exist.  The  range  of  y 
_i  y  -  ci 

Ti  (~F  r"  )  <  0  and 

1  C2C3 

-1  y  “  C1 
T /(--  -  L)  >  y, 

2  C2C3 


must  be  such  that 
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or 


(66) 


y-c 

V  <  v7-ci’ 

and  the  range  of  y  is,  then, 

C, 

- <  y  <  C  (1  +  C  C  ), 

1  '  C?S 

2  *  c1 

and  these  points  are  in  (C^,  ^  -C" * 

C  c 

( — — — • — c,  (l  +  c_c  )]  c  (c. ,  t  q  p 
1  -  c2c^  1  ^  5  23 

Now,  one  can  proceed  to  iterate  the  functional  equation.  When  0  <  x  < 
D(x)  =  pD(C1  +  C2x)  +  qD(C^x)  +  1 

=  pqD(C^ (C^  +  C2x))  +  qD(C jX)  +  1  +  p. 

But 

y  -  c. 


c3(c1  +  c2x)  >  C1C3  >  c2  • 

Here,  it  is  convenient  to  sketch  the  ordering  of  points  and  label  the 
regions  between  discontinuities. 


y  -  c, 


CiC3 


V 


*  ~  c3 

C2C3 


T 

c. 


CjU  +  C2) 


Because  of  Theorems  2  and  3i 
D(x)  =  D(xi) ,  x  e  Xi- 
Then  the  iterated  equation  becomes 

D(x^)  =  pqD(x2)  +  qD(x^)  +  1  +  p 

=  pq2D(xx)  +  qD(xx)  +  1  +  p  +  pq 


d(X;l)  =  1  a  f  »  £a2  = 


+  i 


1  -  q  -  pq  p  (1  +  q) 


(68) 
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Now  each  of  the  arguments  of  the  D(«)  function  is  less  than 

1  (1  “  C2_1) 

k-l^y  Cl  l  _  c  ^ ’ 

C2  2 

so,  again  appealing  to  Theorems  2  and  3»  (71)  may  be  written 


D(x)  =  -1  ~  f  (l  +  qD(x) )  (72) 

q 

and,  finally, 

D(x)  =  1  -kL~  .  (73) 

qp 

The  same  method  applies  when  x  is  in  the  next  interval,  viz., 


(1  -  c*-2) 


(1  -  C^"1)  x_  -  -  , 

_|i_(y  _  - - <  x  <  -^2-(y  -  — j— r~c — 

C2  2  C2  2 
Let  x  =  x^  when  x  is  in 

(1  -  C*"1) 


(0’  „k-l(y  ”  C1  (1  -  c,)» 


and  x  =  x2  when  x  is  in 

,  (1  -  C*'1)  .  (1  -  C*"2) 

(^l(y  "  C1  (l  -  C2)  ^k-2(y  ■  C1  ~  ^k-2)  ^  * 


(1  -  C 


Then  the  functional  equation  becomes 


i  k-i 

D(x2)  =  - (1  +  qD(XjL)), 

,  k-1  .  k 

=  -  U  + 

q  Pk 

-i  k-1 

qp 


(74) 
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The  previous  examples  suffice  to  demonstrate  the  method  for 
finding  D(x)  when  C ^  <  1.  There  are  always  a  finite  number 

of  points  of  discontinuity  between  zero  and  y.  For  some  n, 

C3y  <  T_ny  <  C1  (75) 

and  then  T  ny  is  the  last  point  of  discontinuity  found  by  applying 
inverse  operations,  T^  and  -  Note  that  C^u  <  for  all 

Cl  cxci 

ue[0,  ^ - —]  and  C.  +  C,  <  1,  so  C,u  <  <  C  . 

1  ”  t>2  ^  J  ~  5  —  l  -  1^2  "  1 

The  reader  may  verify  the  fact  that  no  more  than  a  finite  number  of 
inverse  operations  are  necessary  to  assure  that  T  nye(C^y,C^],  using 
the  arguments  of  Section  2  of  the  paper,  "Studies  in  Generalized  Random 
Walks,  I.  Distribution  Functions  and  Moments."  If  there  are  m  points 
of  discontinuity,  exclusive  of  zero,  and  they  are  denoted  by  let 

D(x)  =  D(xJ  for  ±  <  x  <  5^.  Equations  (2)  or  (4)  may  then  be 
used  to  set  up  a  system  of  equations, 

Dtx^  =  f(x1,x2,...xm,b1,b2,...)  (76) 

where  f(...)  is  a  function  of  the  x^'s  and  constants,  b^b^,..., 
which  involve  powers  of  p  and  q.  The  solution  of  (76)  is  then  the 
complete  solution  for  D(*). 


No  simple  method  has  been  devised  for  finding  D(x)  when  C_  +  C,  >  1. 
There  are,  however,  several  approximation  techniques.  These  approxima¬ 
tions  will  be  discussed  in  another  paper  in  connection  with  the  applications. 


The  conditions  that  are  imposed  by  some  of  the  applied  problems  make  the 


solutions  quite  special  and  are,  therefore,  more  logically  discussed 
with  those  problems. 


